Towards a Theory of Natural Scenes

Is it possible to build a science of natural scenes? A science that would make it possible to analyze scenes in a systematic manner, enabling us to classify,understand and predict their behavior. What we have in mind are methods that are sufficiently precise so that they can be implemented successfully by computer code. At first glance it appears unlikely that this would be possible; the variation in natural scenes is immense and it is difficult to see what sort of laws could govern their appearence. To borrow from history of science, it is hard to imagine laws like those of rational mechanics where deterministic, laws, differential equations, exist that enable us to understand and predict the way material bodies move, how they interact with each other. On the other hand, statistical mechanics offers a better paradigm, sacrificing an exact, detailed description of the systems in favor of a statistical representation that only makes probabilistic statement about their behavior. This is what David Mumford has suggested repeatedly, to measure the visual world as it appears to us macroscopically using statistical descriptions. But how can we obtain such probabilities? Does it make sense to describe natural scenes in statistical terms, do they have enough stabilities and invariances for a theory to be feasible? There is a growing literature dealing with this or related questions, mainly of empirical nature,and they have shown some remarkable regularities, see e.g. Huang, Mumford (1999). We shall try to derive analytical, model-based, results.